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ABSTRACT 

This paper reports on two studies based on classroom 
use of episode materials developed by Sociological Resources for the 
Social Studies (SPSS) . The 2-week episodes are intended to introduce 
a sociological perspective into existing social studies courses. The 
studies were designed to explore the relationship, if any, between 
the amount of formal teacher preparation in sociology (i.e., the 
number of hours of exposure to preservice and inservice instruction 
in the discipline) and the effectiveness of the episode materials in 
promoting student learning. Both studies indicate "very little 
difference in student performance, when controlled for ability level, 
regardless of the level of teacher preparation in sociology. ” Among 
the possible conclusions suggested by the comparative studies are: 
’’the materials produced by the SB S3 project can be as effectively 
taught by teachers with considerable sociology and no special 
training as by teachers with considerable sociology and special 
training” (i.e., they are ’’teacher-proof materials”); the effects of 
special training and preparation in subject matter and methods cannot 
be measured in terms of student performance on objective, cognitive 
tests”; and ’’the national evaluation scheme of the SPSS project 
straight jacketed all teachers to such an extent that differences in 
preparation could not affect the results.” (Author/JS) 
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In the world of social studies curriculum projects confidence and fear are 
competing emotions — confidence that the new materials are far superior to the tra- 
ditional texts commonly used, fearful because these efforts may become exercises 
in futility. Their fate lies with the teachers who will ultimately present the new 
packages to students. Why should projects fear what teachers will do? The real 
reason may be a simple manifestation of the so-called "campus chasm" between the 
social science departments and schools of education. Or it may stem from pride of 
authorship; if the materials are so new — so different, can a teacher not involved 
in their development do justice to them? Or this fear may be based on the assump- 
tion that existing materials suffer a dearth of "solid subject matter." (To teach 
the new project materials places excessive demands on teachers with little prepara- 
tion. These teachers have been able to avoid this problem with the skimpy tra- 
ditional treatments.) There is some evidence that high school sociology textbooks 
have this shortcoming.^ And teachers of sociology are probably less-prepared in 



their subject than teachers of the more established disciplines. 

To correct this deficiency, whether imagined or real, efforts have been made 
to encourage more and more summer institutes, in-service institutes, local work- 
shops, etc. The assumption is clear: improving teacher preparation increases 

student learning. To improve teacher preparation means to increase the number of 
hours of exposure to college instruction in the discipline. The increasing number 
of in-service opportunities is designed to implement this assumption. Projects 
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Prepared for delivery at the annual meeting of the National Council for the Social 
Studies, Houston, November 28, 1969. The author acknowledges with thanks the coop- 
eration and advice of F. Lincoln Grahlfs, formerly Supervisor of Evaluation, 
Sociological Resources for the Social Studies. 
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encourage these efforts because, if teachers are as unprepared as some reformers 
think they are, the only alternative is to develop "teacher proof" materials. 
Some projects may have adopted this strategy. None will admit it so none can be 



identified . 



It could be argued that teachers with little subject matter preparation com- 
pensate for this deficiency by superior pedagogical skills. When the projects 
emphasize inquiry, and state that their materials stress the process of knowledge 
acquisition rather than the recall of fact, the "skilled" teacher may be more 
effective than the "knowledgeable" one. Put most social scientists would probably 
agree that teachers with a strong background In social science will probably pass 



on more cognitive learning, especially lower-level factual recall of content, than 
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will teachers with meager preparation. This paper tests that proposition. 

Sociological Resources for the Social Studies is a curriculum project spon- 
sored by the American Sociological Association and funded by the National Science 
Foundation. Since 1964 it has been at work producing short (two-week) units, called 



episodes, which emphasize a sociological perspective and which are intended for use 
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in existing social studies courses. From February of 1967 through June of 1969 
eighteen of these episodes were taught in varying numbers to 13,315 students by 
518 teachers in twenty-two states. Because of the national scope of this evalu- 
ation and the emphasis on sociology this comparison of teacher preparation and stu- 
dent performance was developed. Numbers in some cells were so small that tests of 



significance could not be applied. 
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Although a stratified random sample of classes was sought, the number of sub- 
stitutions and withdrawals eliminated that possibility. The population was, however, 
well-balanced in terms of socio-economic level, geographic distribution, verbal 



ability level, and educational aspiration of students. Types of schools included 
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rural, small urban, suburban, and metropolitan. No private schools tested the 
episodes, and nearly all students were 11th or 12th graders. In each school only 
one class was permitted to evaluate a particular episode. Because schools exer- 
cised autonomy in selecting classes for the evaluation, several episodes which 
appeared to be more difficult were under-represented by lower-abilitv students. 
Eight episodes were evaluated from February to June, 1967, four from October to 
May, 1968, and six from October to May, 1969. 

The purpose of the study was to evaluate student performance on the episodes. 
No attempt was made to secure a sample of teachers representing various levels of 
preparation. Participating teachers were asked to complete a questionnaire which 
included a question about their preparation in sociology. Table One provides 
details on the size of the test population. 

TABLE ONE 

NUMBER OF CLASSES WHICH STUDIED EIGHTEEN UNITS IN SOCIOLOGY, 1967-69 



MEAN VERBAL ABILITY 

LEVEL OF CLASSES PRETEST 1 POSTTEST 2 

(in centile rank 

for 12th grade) Classes Students Classes Students 



75 and above 


74 


1690 


57 


1451 


Ln 

0 

1 


228 


6246 


189 


5278 


30 - 49 


134 


3502 


106 


2823 


below 30 


82 


1877 


54 


1223 


TOTAL 


518 


13,315 


406 


10,775 



Pretest figures include all classes which completed The Psychological 
Corporation's Standardized Verbal Abilities Test. 

Posttest figures include only those classes for which data on teacher 
preparation was available. 
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METHOD 

Every student who studied an episode also completed a standardized verbal abil- 
ities test developed by The Psychological Corporation. Since many classes were of 
mixed abilities, mean verbal ability scores were obtained for each class. Classes 
were then assigned to the appropriate quart ile. The division of classes is seen in 
Table One. In addition to the Verbal Abilities Test, a thirty-five item multiple- 
choice test was administered to both stimulus and control groups. Although some 



attempt was made to include test questions which measured critical thinking abili- 
ties or important concepts of sociology, almost all questions referred to specific 
sociological content mentioned in the episode. Episodes were matched so that stu- 
dents serving as the control group for one episode served as the stimulus group for 
another. Thus the same test was used for pre- and post- measures, but administered 
to different classes. From these pre- and posttest scores class norms were derived 



For each of the four ability levels class pretest norms were multiplied by the num 



ber of students in the class, and the result divided by the total number of students 



at that ability level. This created a mean pretest score for each of the four abil- 
ity levels. 

Teacher preparation in sociology was the most difficult factor to retrieve from 
the evaluation data. Several teachers failed to return a questionnaire or failed to 
answer t:he specific question about their preparation in sociology. Questionnaires 



for the first eight episodes asked the teachers to check one of three categories of 
preparation: "less than three courses," "three or more courses,' sociology as a 

major." Questionnaires for the remaining ten episodes were more specific; they 
asked how many hours in sociology the teacher had completed. With this information 
teachers were sorted into three groups: "less than three courses, three or more 

courses," and "sociology major or equivalent." For this study a course was con- 



sidered to be three hours. Thirty or more hours were considered to be equivalent 
to a major in the field. Table Two shows the number and percent of teachers in 



each of the three categories, 
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This was the weakest link in the study. It was impossible to determine what 
types of courses teachers were reporting. Also, some teachers may have acquired 
over thirty hours of sociology but not elected to major in that field. Conse- 
quently, they would have been placed in the medium preparation category because 
they checked ’’three or more, courses." 



TABLE TWO 

TEACHER PREPARATION IN SOCIOLOGY* 



N % 



Major 


or 


equivalent 


25 


6.16 


Three 


or 


more courses 


192 


47.29 


Less 


than 


; three courses 


189 


46.55 






TOTAL : 


406 





* Based on a test population of social studies teachers 
in twenty-two states, February 1967-June 1969. These 
teachers all agreed to teach a short sociology unit 
in one of their classes. They may have been better 
prepared in the discipline than their colleagues. Al- 
though 518 teachers participated, data on preparation 
in sociology was supplied by only 406. 

Data on teacher preparation was available for four hundred and six of the 518 



participating teachers. Each of these were then assigned to one of twelve cells 
representing three levels of teacher preparation and cross-cut by four levels of 
mean verbal ability. Posttest norms for each class were multiplied by the number 



of students in that class and the total for each cell divided by the number of stu- 
dents in that cell. The pretest norm for each ability level was subtracted from 



the posttest norm for each cell at that level. This produced a gain for each level \ 



of teacher preparation according to mean verbal ability level of students. The 



appendices indicate scores and gain for each of the eighteen episodes; Table Three 
is a composite of all episodes. When a pretest norm for an ability level in a 
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particular episode was based on an N of less that! three classes and/or fifty stu- 
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dents the gain was not included in the composite. J 

i 

RESULTS 1 



Twenty-five teachers (6.16%) reported having a major or equivalent in socio- 
logy. Fifteen of these twenty-five taught classes at the second highest verbal 
ability level, the remaining ten being divided among the other levels. Any com- 
parisons would, therefore, be questionable. The fact that seven of the twelve 
cells in Table Three had less than thirty cases indicates a lack of statistical 
significance. Keeping these low numbers in mind, the results show a weak positive 
relationship between teacher preparation and student performance. 

In three of the four ability levels classes of teachers with highest prepar- 
ation showed a greater gain than other classes at the same level. Classes with 
teachers of medium preparation level tended to show less gain than other classes. 
But the differences were small. Table Four provides a comparison of gain accord- 
ing to teacher preparation. 

Classes taught by teachers at the highest preparation level showed the highest 
gain at every ability level but one. The greatest differences were with classes 
in the middle ranges of verbal ability. This finding may be attributed to the 
greater number of cases at those levels; the upper and lower ability levels had 
only two and three classes, respectfully, taught by the more prepared teachers. 

The second lowest gain of all twelve categories was for high ability classes taught 
by highly prepared teachers. But the differences were small and a larger popula- 
tion might change that finding. 

Classes with teachers having a medium level of preparation in sociology 
tended to show the lowest gain. Differences between these classes and those 
taught by teachers with low levels of preparation were almost nil, however. The 
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TABLE THREE 



STUDENT PERFORMANCE ON UNITS IN SOCIOLOGY 
ACCORDING TO TEACHER PREPARATION IN SOCIOLOGY 



MEAN VERBAL ABILITY 
LEVEL OF CLASS 



TEACHER PREPARATION IN SOCIOLOGY 



(in centile 


rank 


Major or 


Three or 


Less than 


for 12th grade) 


equivalent 


more courses 


three course; 




Episodes 


2 


13 


16 




Classes* 


2 


26 


29 


75 and above 


Students 


30 


702 


719 




Gain^ 


2.28 


4.67 


4.45 




Episodes 


10 


18 


18 




Classes* 


15 


90 


84 


50 - 74 


Students 


435 


2464 


2379 




Gain^ 


4.16 


3.72 


3.81 




Episodes 


4 


16 


16 




Classes* 


5 


54 


47 


30 - 49 


Students 


165 


1416 


1242 




Gain^ 


6.29 


2.61 


2.72 




Episodes 


3 


1.1 


15 




Classes* 


3 


22 


29 


below 30 


Students 


77 


481 


665 




Gain^ 


3.92 


1.72 


3.02 



% 



1 






Although in three of the ability levels there were only five or fewer classes 
(and episodes) taught by the most-prepared teachers, these were included for 
comparison. This small N should be considered in any comparisons. 



2 - 

Posttest norm of classes in this cell minus pretest nqrm of classes for this 

verbal ability level. 
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TABLE FOUR 



DIFFERENCES IN STUDENT GAIN AFTER STUDYING UNITS IN SOCIOLOGY 
(BY LEVEL OF TEACHER PREPARATION IN SOCIOLOGY) 



MEAN VERBAL ABILITY COMPARISON OF STUDENT GAIN* 

LEVEL OF CLASSES 
(in centile rank 
for 12th grade) 



Major or equivalent compared with less 
than three courses 

-2.17 
+ .35 
+3.57 
+ .90 



Major or equivalent compared with three 
or more courses 

- 2.12 
+ .44 
+3.68 
+ 2.20 





Three or more courses as compared with 
less than three courses 


75 and above 


- .05 


50 - 74 


- .09 


30 - 49 


— .11 


below 30 


- 1 . 3 d 



75 and above 
50 - 74 
30 - 49 
below 30 



75 and above 
50 - 74 
39 - 49 
below 30 



* Gain measured by subtracting pretest norms for verbal ability level from 
posttest norms for the same ability level, according to level of teacher 
participation. 
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average difference on a forty question test was .387, little more than 1/3 of a 
question. The fact that the medium preparation group was at every level slightly 
below the low preparation group suggests that a little knowledge may or may not be 
dangerous but it appears, in this case, to be useless. There w as a general ten- 
dency toward a not surprising conclusion; the higher the verbal ability level of 
the class, the greater the gain. That this is true despite the teacher preparation 
differences suggests a number of conclusions about the x^isdom of encouraging 
increased teacher preparation. The most important statement that can be made is 
simply that there was in fact very little difference in student performance when 
controlled for ability level regardless of the level of teacher preparation. 

Classes with teachers of medium and low preparation levels showed gains in 
direct relation to ability level. That is, the higher the ability level the greater 

! 

I the gain. 

; 

[, 

\ 

| CONCLUSIONS 
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Perhaps the episodes are nearly teacher proof. The small number of teachers at 
the highest preparation level, the direct relationship of student ability to student 
gain between classes with teachers of medium preparation level and those of minimal 
preparation suggest this possibility. Since every teacher knew they were evaluating 
experimental materials they may have attempted to follow the schedule and suggested 
procedures very carefully. Comments on teacher questionnaires support this sugges- 
tion. Now that some of the materials have become commercially available it may be 
that teachers will be more willing to experiment, deleting parts of episodes and 
modifying material. The question of "teacher proof" materials cannot be answered 
yet. But individual cases suggest the unlikelihood of this possibility. 

A more reasonable conclusion is that something more than additional study in 
subject matter is needed to implement new curriculum materials. The sociology 
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presented by the project materials differs markedly from that commonly taught in 
undergraduate sociology courses. The pedogogical approach is also quite different 
Emphasis on workshops and institutes which attempt to do nothing more than 



increase teacher background in sociology will likely result in little improvement 
in student learning on new sociology materials. 

When similar findings were reported last year at the AERA meetings it was 
suggested that the findings were consistent with the goals of the project. That 
is, the episodes should be. so designed as to allow almost anv teacher to easily 
integrate them into existing courses. Because few teachers have a high level of 



preparation in sociology the episodes should not fare better with them. The epi- 
sodes are, after all, not designed to teach sociology but to add a sociological 



perspective to existing courses. It was further suggested that teacher prepara- 
tion in sociology might make a difference in a full semester sociology course. 
Recent evidence does not support this notion. 



Does in-service preparation of teachers specifically oriented toward new 
curricular materials enhance the success of those materials in the classroom? 



Do teachers who have recently acauired increased levels of preparation in soci- 
ology pass on more of this knowledge to their students? Last spring the SRSS 
one-semester sociology course was taught bv two hundred and twentv-two teachers 



to a like number of classes. T shall not dwell on the evaluation mechanism— 



that is Mr. Grahlfs’ concern. His paper has given you an accurate picture of 



that experience. T would like to add, bovrever, one point. There were four types 
of teachers involved in that evaluation: a small group which had a close work- 



ing relationship with the development of the course, a larger group which knew 
nothing of the course before they received it, and two groups who had special 
training in the teaching of the course. The latter can be subdivided into those 



teachers who attended summer institutes and those who were enrolled in a in-service 



institute while they were teaching the course. No pretest was administered so a 
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comparison of gain is impossible. But a comparison of raw scores can offer some 
indication of any differences according to teacher preparation. 

Classes were administered the same verbal abilities test given students who 



studied the episodes. Class norms were figured and the entire group was organized 
by quart iles. Those teachers with a close working knowledge of the course were 
excluded from this comparison for obvious reasons. 

Three levels of teacher preparation were compared: those who attended 

summer institutes (48) , those who attended in-service institutes (50) , and those 
who attended no institute and had no more than five courses in sociology (35) . 

The last group, incidentally, was composed mainly of teachers with fewer than tftree 
courses in sociology. 



s 



Table Five compares the three, controlled for ability level. In only one 
instance did teachers with no special training and little sociology fare differ much 
from their better prepared colleagues. That was at the fourth ability level; 
although only two classes were reported in that category, they are. compared with 
three classes with teachers who attended summer institutes and four classes with 



it 



teachers who attended in-service institutes. At that ability level there is a 
marked difference between the results from classes with summer institute teachers 



and with classes of naive teachers (nearly seven points out of fifty). With this 



exception the differences among the three teacher preparation groups are almost 
totally insignificant. For the record, at the second and third ability levels 
those classes whose teachers had no institute experience achieved slightly higher 
results on the final examination than did those classes whose teachers had attended 



one or the other types of institutes. And at the highest ability level classes 
with naive teachers still did better than classes with teachers attending on-going 
institutes while the course was being taught. 



Put less politely, it seems that the final score on a fifty-item objective test 
administered at the end of a course is likely to differ very little if teachers had 
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TABLE FIVE 



COMPARISON OF STUDENT SCORES ON A FIFTY-ITEM FINAL TEST IN SOCIOLOGY 
ACCORDING TO TEACHER PARTICIPATION IN SOCIOLOGY INSTITUTES 



ABILITY LEVEL 
OF CLASSES 



SU MMER INSTITUTE 

( 48 ) 



IN-SERVICE I NS TITUTE 
(50) 



NO INSTITUTE 



(35) 



Classes 
Students 
Average Score 



18 

498 

29.45 



13 

370 

28.26 



9 

214 

29.17 



II 



Classes 
Students 
Average Score 



20 

546 

25.08 



26 

999 

25.43 



17 

491 

26.03 



III 



1 



3 * 
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Classes 7 77 
Students 167 188 184 
Average Score 21.11 21.46 22.61 



IV 



Classes 3 42 
Students 51 102 46 
Average Score 23.16 19.28 26.51 



acquired special training in teaching that course. At least we can say this based 
on our tests and those institutes centered on our materials , One could fault the 
institutes though I think they are probably no better or no worse than those from 
other disciplines. One could fault the test. I shall do neither. But I would 
like to point out that it might very well be that the test did not measure the 
learnings which the institutes prepared teachers to impart. That is a hunch — 
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not an assertion. 
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There are at least three major conclusions which can be drawn from this crude 
comparison: the materials produced by the SRSR project can be as effectively 

taught by teachers with little sociology and no special training as by teachers 
with considerable sociology and with special training; the effects of special 
training and preparation In subject matter cannot be measured in terms of student 
performance on objective, cognitive tests; and finally that the national evaluation 



scheme of the SRSS project straight jacketed all, teachers to such an extent that 



differences in preparation could not affect the results. It should be added that 
the division of classes into ability levels crosscut by teacher preparation levels 
reduces the N so much that significant statements cannot be made. Further, all 



conclusions reached are based on a comparison of results on a nonstandardized and 
never before administered examination. These disclaimers provide the author with 
a needed escape mechanism. 
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NOTES 



In 1966 the five secondary school textbooks in sociology which were 
then available were analyzed. Fifteen topics which most sociologists 
would consider central to their discipline were considered. The per- 
cent of total subject matter pages devoted to those topics was report- 
ed for each text. A few examples: only one text gave stratification 

over five percent (5.1); the text having the most subject matter pages 
on social change accorded that topic 4.0 percent? the scientific method 
nevei? received over 2.8 percent. The four topics which received the 
most extensive coverage in at least one of the texts were: education 

(20%) ? human ecology/population (14%) ? culture (13.6%) ? and the fami- 
ly (13.2%). Three of these four reported coverages were in one book. 
(William M. Hering, Jr., "An Analysis of the Five Secondary School 
Textbooks in Sociology?" Sociological Resources for the Social Studies, 
Ann Arbor, Michigan, 1966.) 



The one hundred and forty-nine teachers of the first eight episodes 
were divided into two groups: teachers of sociology courses and teach- 

ers of non-sociology courses. 9.2 percent of the former had majored 
in sociology, 64.6 percent reported having three or more sociology 
courses, and 26.2 percent reported taking less than three courses in 
sociology. When further divided according to teaching experience, only 
5.9 percent of the sociology teachers who had taught five or more years 
had majored in the field, although 21.4 percent of the less-experienced 
teachers had sociology majors. (William M. Hering, Jr., "Sociology 
and the High School: What and why?" The Indiana Social Studies 

Quarterly , Ball State University, Muncie, Indiana, Vol. XX, No. 3, 
Winter, ±967-8, p. 7.) 



One sociologist concerned with the fate of his discipline in the high 
schools wrote, "Many secondary school sociology teachers are poorly 
trained to teach sociology: they have been teaching history, social 

studies, or government for several years and may have taken on socio- 
logy when the principal wanted to add it to the curriculum as a half- 
year senior course. Consequently, they need help." He notes that 
"these observations result from interviews with .high school sociology 
teachers. They are based upon impressions, rather than quantitative 
assessments..." (Abbott L. Ferriss, "Secondary Sociology Teachers," 
unpublished manuscript, 1968) . 

SRSS is also producing a one-semester sociology course and a number of 
readings in sociology for high school teachers. Specific information 
about all of its activities can be obtained from SRSS headquarters, 

503 First National Building, Ann Arbor, Michigan 48108. 

In a review of research on teacher preparation, Seymour Metzner of the 
City University of New York states, "The plain fact is that there is 
not a single study that, after equating for pupil intelligence and 
socio-economic status, has found the length of teacher preparation 
variable to be even peripherally related to pupil gain, let alone 
being of major importance in this educational outcome." (Seymour 
Metzner, "The Teacher Preparation Myth: A Phoenix Too Frequent," Phi 

Delta Kappan, Vol. L, No. 2, October, 1968. p. 106.) Support for this 
position can be found in Eugene Auerbach, "Liberal Arts Opposition to 
Professors of Education," School and Society , November 21, 1959, pp. 
473-4, and Donald P. Hoyt, "College Grades and Adult Accomplishment: 

A Review of Research," E ducational Record , Winter, 1966, pp. 70-5. 






